Skip to content

feat: add VariantArrayBuilder::with_shredding#9963

Open
Shekharrajak wants to merge 1 commit into
apache:mainfrom
Shekharrajak:feat/variant-array-builder-shredding
Open

feat: add VariantArrayBuilder::with_shredding#9963
Shekharrajak wants to merge 1 commit into
apache:mainfrom
Shekharrajak:feat/variant-array-builder-shredding

Conversation

@Shekharrajak
Copy link
Copy Markdown

@Shekharrajak Shekharrajak commented May 12, 2026

Ref #8480

Rationale for this change

Adds a builder method that configures VariantArrayBuilder to produce a shredded VariantArray at build() time.

What changes are included in this PR?

Add shredding_schema: Option field to VariantArrayBuilder
Add with_shredding(as_type: DataType) -> Self — rows are still appended unshredded via append_variant; the shredding pass runs once at build() by delegating to shred_variant

Are these changes tested?

unit tests

Are there any user-facing changes?

Yes, usage:

let arr = VariantArrayBuilder::new(3)
    .with_shredding(DataType::Int64)
    .tap(|b| { b.append_variant(Variant::Int64(42)); })
    .build(); // returns shredded VariantArray

@github-actions github-actions Bot added the parquet-variant parquet-variant* crates label May 12, 2026
@Shekharrajak Shekharrajak force-pushed the feat/variant-array-builder-shredding branch from afac072 to f52d0c9 Compare May 12, 2026 05:01
@Shekharrajak
Copy link
Copy Markdown
Author

@alamb @scovich , please have a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant